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ABSTRACT 


A magnetic  tape  conversion  program  is  being 
developed  to  facilitate  picture  processing  by  medium 
sized  digital  computers.  The  approach  is  to  obtain 
a revised  image  format  that  allows  sophisticated  al- 
gorithms to  process  small  contiguous  areas.  A tree 
structure  is  presented  based  on  regular  decomposition 
of  two-dimensional  arrays.  Magnetic  tape  capacity 
parameters  are  reviewed  and  applied  to  develop  the 
best  tape  organization  for  this  data  structure.  Al- 
ternate neighbor  region  and  partitioning  regimes  are 
described  and  a sample  calculation  discussed. 


1.  INTRODUCTION 

Digitized  picture  consists  of  large  sequential 
files  generated  by  scanning  devices.  Magnetic  tape 
is  the  most  common  storage  medium  for  such  files 
since  it  is  portable  and  possesses  sufficient  stor- 
age capacity  to  contain  an  entire  visual  image.  One 
common  resolution  for  digitized  images  (1024  x 1024) 
results  in  over  a million  data  points:  light  samples 

found  in  scanning  over  the  1024  lines  (raster  scan) 
can  have  grey  levels  ranging  from  0 to  255.  Each 
value  corresponds  to  a spacial  picture  element  or 
pixel . Tape  storage  for  pixels  quantized  to  756 
levels  uses  three  characters,  one  for  each  numeric 
in  the  values  000  to  255.  Pixels  are  represented 
then  by  triples  of  characters  where  each  character 
is  a byte  of  5 or  8 bits  (depending  on  tape  device). 
When  pixels  are  stored  using  triples  of  characters, 
files  of  over  3 mega-bytes  are  formed  for  a single 
image.  Since  25  mega-bytes  fit  on  a single  magnetic 
tape,  the  raster  scans  of  several  pictures  can  be 
contained  and  accessed  sequentially.  In  this  paper 
we  address  the  problem  of  using  this  large  storage 
capacity  to  orqanize  the  visual  information  in  a 
single  image  for  future  processing. 

The  goal  of  this  research  is  converting  raster 
scan  tape  records  of  images  to  a series  of  smaller 
sets  of  pixels  restricted  to  subregions  of  the  ori- 
ginal. This  extends  / 1 / and  creates  hierarchial 
organizations  / 2/  of  visual  data.  The  subregions 
may  be  arranged  in  a tree  and  obtained  by  regular 
decomposition  of  the  original  image  /3/.  The  form 
of  the  tree  (some  regions  may  be  deleted  as  value- 
less) could  yield  object-location  and  symmetry  de- 
tection information  /3,4/. 

This  paper  describes  the  current  status  of  an 
effort  to  develop  a computer  program  system  to  con- 
vert raster  scan  records  to  hierarchical  form  invol- 
ving many  smaller  images.  The  planned  organization 
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of  the  resultant  tape  lwaye  is  described  with  detail- 
ed analysis  of  tape  capacity  and  pointer  definition 
requirements.  A summary  of  considerations  regarding 
choice  of  image  decomposition  mode  is  presented. 
Finally,  a display  program  to  generate  the  picture- 
tree  on  an  interactive  graphics  console  is  discussed 
and  a sample  tree  shown. 


2.  TAPE  ORGANIZATION  OF  IMAGES 

The  'raster  scan'  organization  obscures  actual 
physical  proximity  of  picture  elements.  However, 
the  limits  on  fast  memory  storage  reguire  processing 
small  areas  of  an  image.  For  example,  see  /5/;  there 
locating  the  boundary  of  single  chromosomes  involved 
extensive  computing  on  successive  image  lines:  only 

when  that  task  was  done  could  any  information  be  ob- 
tained. This  section  discusses  now  regional  proxi- 
mity of  pixels  can  be  obtained  by  decomposing  a di- 
gitized image  to  raster-scan  sub-pictures.  The  key  to 
to  the  technique  is  maintaining  an  accurate  file 
directory  to  facilitate  efficient  retrieval  of  pic- 
ture regions. 

Picture  files  are  usually  segmented  into  blocks: 
each  block's  byte  store  light  intensity  values  in 
pixels  corresponding  to  one  line  of  the  image.  Large 
blocks  generally  reduce  I/O  time,  but  several  lines 
may  not  fit  in  fast  memory  simultaneously.  A medium 
size  computer  with  48K  bytes  of  core  storage  can  hold 
only  16  lines  (1.5%)  of  a 1 mega-byte  digitized  im- 
age, even  if  all  available  fast  memory  is  used  (and 
this  leaves  no  room  for  programs).  To  examine  even 
a small  image  region,  this  could  cause  extensive  I/O 
costs.  Reducing  storage  requirements  for  picture 
data  is  necessary  for  image  processing  on  medium 
size  machines.  Before  describing  how  we  plan  to  do 
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Figure  1.  Magnetic  tape  storage  capacity. 
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Figure  2.  Percent  of  tape  capacity  used 
by  one  mega-byte  file. 

this,  we  need  to  examine  tape  storage  capacity  in 
detail . 

2a.  Tape  Capacity 

If  C is  tape  capacity  (maximum  number  of  bytes 
of  data  that  can  be  written  on  tape); 

L is  the  tape  length  in  inches; 

B is  the  block  size  in  bytes; 

D is  the  density  in  bits  per  track'; 

2 

I is  the  interblock  tape  qao  length  in  inches  ; 

then  the  storage  capacity  of  magnetic  tape  is: 

C = (L  * B)/  ( (B/D)  + I)) 

Standard  10.5"  9-track  tape  reels  hold  2400  feet 
(23,800")  of  tape,  and,  when  recorded  at  a block  size 
of  3K  bytes  with  a density  of  800  bpi,  will  hold  19.2 
mega-bytes  of  data. 

A single  image  stored  sequentially  in  conven- 
tional raster-scan  format  consists  of  over  3 mega- 
bytes. Such  a linear  file  is  efficient  for  picture 
display  because  of  the  line-at-a-time  (flying  spot) 
nature  of  CRT  devices.  However  the  19.2  mega-bytes 
can  be  used  to  facilitate  retrieval  and  matching 
parts  of  image  data,  and  that  is  the  central  aim 
of  this  research. 

Density  of  200,  556,  800,  and  1600  bpi  are  standard 
hardware  recording  modes  for  digital  tape.  Fiqure  1 
illustrates  the  available  storage  for  standard  7 and 

9 track  tape  reels  as  a function  of  density  when  re- 
cording with  a block  size  of  IK  bytes'*.  At  556  bpi 

10  mega-bytes  of  information  can  be  recorded  on 

either  tape:  room  enough  for  10  one  mega-byte  image 

1.  Usually  200,  556,  800  or  1600  per  track. 

2.  This  is  0.6"  for  9-track  tapes  and  0.75"  for  7- 
track  tapes. 

3.  Larger  blocksizes  will  increase  capacity  by  re- 
ducing the  occurrence  of  interblock  gaps. 


A 

files.  This  is  further  illustrated  by  Figure  2 
which  shows  the  percentage  of  available  storage  used 
by  a one  mega-byte  file  on  both  7 and  9 track  tapes. 
In  the  following  sections  we  show  this  extensive  tape 
storage  capacity  is  soon  exhausted  if  storaqe  main- 
tenance is  not  enforced  when  reorganizing  picture 
data. 

2b.  Data  Reorganization  and  Pointer  Definitions 

Image  data  tape  files  generally  have  blocks 
correspond  to  one  scan  line  and  triples  of  characters 
indicating  pixel  light  intensity  (both  occur  in  /6, 
7/).  Since  typically  15%  of  available  tape  storage 
is  needed  for  a picture,  refinement  by  regular  de- 
composition /l,  2,  3/  can  be  implemented  by  dupli- 
cating pixel  intensities  on  tape  in  shorter  raster- 
scan  lines  than  in  the  original  image.  The  result- 
ing series  of  sub-pictures  is  stored  after  the  whole 
image's  last  row  last  pixel.  If  sub-pictures  at  the 
same  level  do  not  overlap,  a single  decomposition  re- 
sults in  creation  of  a reorganized  "duplicate"  set 
of  data,  essentially  the  same  length  as  the  original 
digitized  image  (see  3. a for  "level"). 

Picture  refinement  creates  new  sub-pictures 
stored  on  tape  and  additional  leaves  in  the  tree 
structure  representing  the  image.  Each  node  on  the 
growing  structure  will  contain  its  identifier,  fea- 
ture measurements  over  the  area  (i.e.,  average  pixel 
intensity,  measure  of  gradient  change,  etc.)  and 
pointers  to  contiguous  data  on  tape  as  a well-defined 
sub-picture.  Dedicated  tape  storage  is  inherent  to 
the  creation  of  sub-pictures  and  their  relative 
positions  are  approximated  by  the  tree-structure 
(each  sub-picture  corresponds  to  a node  in  the  tree). 


figure  3.  Information  stored  in  data  structure  nodes. 


Figure  3 shows  information  fields  in  each  struc- 
ture node.  The  node  identification  fields  contain 
information  to  identify  ancestors  and  level  of  de- 
composition. Each  structure  node  contains  four  mea- 
surement fields  for  sub-picture  feature  data  and 
four  pointer  fields  used  to  locate  the  stored  sub- 
picture on  tape. 

Regular  decomposition  of  picture  data  into  sub- 
quadrants has  been  extended  by  /8/  to  include  neigh- 
bor quadrants  that  may  possibly  contain  extensions 
of  an  object  in  the  quadrant  which  is  to  be  isolated 
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for  finer  picture  decomposition.  Data  reorganization 
on  tape  could  also  include  border  information  at  each 
refinement.  Overlap  of  data  may  reduce  tape  "thrash- 
ing" if  border  information  is  found  to  be  essential 
for  object  recognition.  In  Figure  4 storage  require- 
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Figure  4.  Number  of  Pixels  to  store. 


ments  for  successive  refinements  of  image  data  by 
quadrants  and  ninths  is  shown  to  increase  exponent- 
ially when  border  data  is  included  in  the  replica- 
tion of  the  file.  Including  border  data  expands  the 
boundary  of  each  sub-picture  to  the  midpoints  of 
each  neighboring  partition;  border  inclusive  parti- 
tions for  quarters  result  in  a file  2.25  times  lar- 
ger than  the  original  sub-picture,  3.5  times  larger 
when  partitioning  in  ninths  (see  Figure  4).  If  only 
10%  of  the  tape  is  used  to  store  the  original  image 
(N  in  Figure  4 equals  0.1  of  available  storage) 
three  full  refinements  by  quadrants  will  fill  the 
tape  when  border  data  is  included  and  only  two  full 
refinements  will  fill  the  tape  when  refined  by  ninths. 

Additional  refinement  is  possible  hy  increasing 
tape  storage  capacity,  compressing  the  representation 
of  the  image  (reducing  N),  or  by  selecting  only 
"promising"  sub-pictures  for  storage  during  refine- 
ment. 


The  first  alternative  is  hardware  dependent 
making  it  the  least  expedient  accommodation  for  most 
computer  installations.  Our  computer  system  uses 
7-track  tape  drives  of  556  bpi  density  making  the 
storage  capacity  for  standard  reels  11.378  mega-bytes. 
This  hardware  restriction  directs  our  attention  to 
the  latter  two  alternatives  for  increasing  image  re- 
finement capability. 

We  plan  to  compress  image  data  by  a linear  map- 
ping of  the  triples  of  characters,  which  represent 
pixel  grey  codes  (0-255),  to  single  byte  encoding. 

Each  6-bit  encoding  forms  a one-byte  0-63  grey  scale 
representation  for  pixels  that  previously  used  3 


bytes  of  tape  storage.  This  reduces  N,  the  amount 
of  storage  needed  on  tape  for  each  image.  Despite 
our  low  density  tape  facility  (i.e.,  7-track,  556 < 
bpi),  N is  kept  to  9.21%  of  available  storage.  We 
sacrifice  the  reduced  grey-level  discrimination  in 
order  to  increase  tape  storage  capacity.  Selecting 
"promising"  sub-pictures  is  discussed  in  sections 
3b  and  4. 


3.  DECOMPOSITION  OF  IMAGES 

The  following  presents  in  a condensed  format 
concepts  relating  to  regular  partitioning  (regular 
decomposition,  tree  data  structure,  pyramid  data 
structure  /9/).  An  example  is  given  after  the  basic 
terms  developed  in  /l,  2,  3,  8/  are  introduced. 

REFINEMENT  OF  6 x 6 ARRAY 
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figure  5.  Ninth  partition  and  quarters  pf  r“sult. 


Figure  6.  Quarter 
partition  and  ninth 
of  result. 


3a.  Sub-Picture  Concepts  and  Terms 

Definition  1 : A sub-area  one-quarter  or  one-ninth 

of  a digitized  picture  or  any  previous  quarter  or 
ninth  of  it,  is  called  a quadrant. 

Lemma  1:  The  smallest  pixel  array  that  is  refinable 

by  either  quarters  or  ninths  is  of  size  6x6,  and 
there  are  neighboring  pixels  on  every  two-quadrant 
boundary  that  can  be  organized  into  single-pixel 
neighbor-subquadrants. 

Proof:  Refinement  into  quarters  requires  bisection 

of  the  pixel-array  sides,  and  into  ninths,  tri- 
section. The  smallest  integer  divisible  by  both 
two  and  three  is  six.  The  neighboring  pixels  ad- 
joining a ninth-partition  quadrant  are  obtained  as 
quarter-partitions  of  the  adjacent  (ninth)  qua- 
drant (see  Figure  5).  Those  adjoining  a quarter- 
partition  quadrant  are  obtained  as  ninth-partitions 
of  the  adjacent  (quarter)  quadrant  (see  Figure  6). 
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Definition  2:  In  a pixel  array  of  any  size  (includ- 

ing an  entire  digitized  picture),  the  root  of  a 
tree  corresponds  to  the  entire  array,  and  successor 
tree  nodes  to  its  regular  partitions.  The  root  is 
called  level  one  and  quadrants  resulting  from  fur- 
ther subdivision  are  at  one  more  level  than  the 
array  they  divide. 

Definition  3:  Quadrants  with  one  higher  level  value 

and  adjacent  to  a quadrant  are  called  neighbor  sub- 
quadrants. 

Lenina  2:  The  size  6x6  pixel  array  is  the  smallest 

that  can  be  regularly  partitioned  twice  to  yield 
neighbor  subquadrants  for  both  quarter  and  ninth 
first  level  refinements. 


Proof:  See  proof  of  Lemma  1 and  Figures  5 and  6. 


REFINEMENT  OF  4 x 4 ARRAY 


Figure  7.  Three  levels 
of  a 4x4  array:  Entire 
' area  to  single  pixel . 
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Figure  C.  Convention  for  quadrant  labels,  directions. 


Definition  4j^  Direction  information  refers  to  spa- 
tial  relationships  that  can  he  extracted  from  a 
tree  representing  an  image.  (Referring  to  Ficure 
8,  presence  of  quadrants  A and  D in  a tree  anc 
absence  of  B and  C,  implies  downward-diagnol 
orientation  of  an  object  in  the  image.) 


Lemma  3:  Both  6x6  and  4x4  pixel  sets  can  support 

three-level  picture-area  comparison  algorithms. 

The  tree  of  quadrants  retained  can  provide  some 
eight-direction  information  on  object  location  nd 
orientation  in  either  case. 

Proof : By  examination  of  Figures  5 through  8.  Note 

that  6x6  arrays  are  shown  with  alternate  ninth 
and  quarter  partitioning  (following  / 7/ > while  the 
4x4  array  in  Figure  7 shows  only  quarter  parti- 
tions. 


Figure  9.  Stylized  airplane  subpictures. 


3b.  Sample  Storage  Computation 

During  picture  refinement  (and  data  structure 
creation)  sub-pictures  can  be  ignored  (not  stored  on 
tape)  or  selected  for  further  refinement  on  the  basis 
of  their  feature  measurements.  Figure  9 is  used  to 
illustrate  sub-picture  selection  using  a simple  aver- 
age intensity  threshold  criterion.  Figure  9(a)  shows 
the  original  image  as  a single  picture  before  refine- 
ment. Figure  9(b)  shows  the  skeleton  of  successive 
partition  lines  imposed  by  hand  simulating  regular 
decomposition.  The  shaded  area  in  Figure  9(c)  is  the 
portion  selected  by  decomposition  to  be  stored  on  tape 
thereby  reducing  image  data  and  isolating  informative 
regions. 

4.  CONTROL  PROGRAM  ORGANIZATION  AND 
PICTURE  TREE  DISPLAY 


Image  reorganization  requires  accurate  book- 
keeping operations.  The  data  utility  and  control  pro- 
grams execute  this  and  are  designed  to  enable  flexi- 
ble interaction  with  the  new  form  of  image  data.  The 
programs  are  like  an  operating  system  since  a variety 
of  computer  facilities  are  integrated  in  our  installa- 
tion (including  SDS  920,930  and  IMLAC  computers). 

Human  interaction  is  allowed  any  time  between  tasks; 
either  automatic  or  human  control  of  image  refinement 
and  feature  extraction  is  possible;  display  of  image 
areas  on  peripheral  devices  is  provided.  Our  goal  is 
a tool  for  operator  use  to:  (1 ) obtain  informative 
picture-regions,  (2)  examine  the  syntactic  relation- 
ships described  by  the  tree  structure  and,  (3)  provide 
indexing  for  image  data  retrieval.  The  creation  and 
maintenance  of  the  tree  structure  is  essential  to  our 
system. 

Figure  3 illustrates  information  fields  in  a 
typical  structure  node.  Each  node  uses  7 SDS  24-bit 
words.  In  addition  to  an  abstract  image  representation 
the  structure  acts  as  a directory  for  accessing  pre- 
viously refined  regions  or  finding  quadrants  that  c?n 
be  refined  to  yield  sought  sub-pictures.  During  pro- 
gram operation  IK  of  core  is  allocated  to  the  data 
structure  leaving  room  for  146  7-word  nodes.  In 
effect  this  enablos  3 full  refinements  of  a picture 
by  quadrants  (i.e.,  4 + 16  + 64  = 84  nodes  after  3 
refinements).  During  picture  refinement  feature  mea- 
surements (i.e.,  average  intensity,  gradient  measures, 
etc.)  can  be  gathered  simultaneously  as  tape  data  is 
reformated  to  picture  partitions.  Picture  refinement 
programs  are  I/O  bound,  the  time  used  in  transmitting 
data  to  and  from  tape  is  often  the  slowest  portion  of 
any  tape  accessing  program.  As  data  is  transmitted 


from  one  part  of  the  computer  processing  can  continue 
in  another.  This  means  no  significant  timing  costs 
are  introduced  when  features  are  extracted  if  they 
are  processed  during  refinement  and  syncronized  care- 
fully with  tape  access  instructions.  Information 
gathered  during  refinement  may  guide  the  program  or 
operator  to  focus  further  refinement  on  those  areas 
with  conspicuous  feature  measurements. 


In  future  work  we  will  experiment  on  /6,7/  to 
evaluate  partitioning  schemes  (/l,  2,  3,  8/)  and 
neighbor  zone  retention  rules  i/8/,  above  illustra- 
tions). ; 
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